| Review 1 | |
| PC member | Camille Gobert |
| Overall evaluation |
The submission discusses the relationships that exist between substrates and accessibility. The
authors demonstrate that the requirements for providing accessible software to end-users, such
as accessibility trees, also provide a technical solution for extending and composing arbitrary
applications. To support this claim, they present Allio, an experimental system that (1) helps
developers read, watch, and write accessibility trees and (2) lets them display tools that let
users interact with the tree's data, such as a global search-and-replace tool. In addition, the
authors repeatedly highlight that they were able to develop Allio only because others previously
fought for making software more accessible, resulting in standard practices and legal
constraints that explain why accessibility trees are so ubiquitous in today's software, even
though they were not designed to maximise the profits one can make out of incompatibility. In
line with, e.g.,
Church and Samosir's submission from last year's Substrates workshop, this
submission challenges the goal of research on substrates, which, they argue, should not only
consider solving technical issues (such as interoperability and reactivity) but also question
how substrates may or may not fit within a particular socio-economical system and what political
changes may be needed to make this model of computing sustainable in the long run. As a supporter of postmodern approaches to substrates, I definitely agree with most of the authors' arguments, and look forward to seeing Allio in action! I also appreciate that the submission emphasises the tension between what several of us seek in substrates (interoperability, malleability, etc.) and what is or is not sustainable under our current economic system. I therefore support the authors' call to reflect on how we might position ourselves (to the point of taking political stances?) and act in favour of the digital ecosystem we all want to realise. I have a few more comments/questions: one is about the framing, another one is about defining/nuancing the concept of substrate, and the rest is about Allio. 1. I disagree with the statement "disability rights movements produced the only infrastructure in contemporary computing that organizes software by what things are rather than who made them" (§1, ¶4). For example, I would argue that the long-lasting and ubiquitous standards that power web pages were not, as far as I know, produced by this movement. Nevertheless, they do provide a standard vocabulary for accessing and interacting with every web page: for example, HTML and the DOM seem pretty similar to accessibility trees—perhaps even more so, given the variability of accessibility features across operating systems (as the authors mention later in the paper). 2. I find value in the authors' framing of their technical proposition as a "foundation" (§3, ¶6) for working on/with substrates and of Allio as a "proto-substrate" (§4, ¶1), which they relate to several principles that were presented during last year's Substrates workshop, such as those of Edwards, Clemens and Basman, as well as with the modern vs. postmodern distinction that Kell and myself pointed out. Since the authors oppose two ways to qualify software infrastructures ("app-centric" and "material-centric", §4, ¶1), and since at least one other submission (Clemens and Basman's) includes a sort of design space of substrates, it made me wonder whether we could shift the difficult goal of coming up with a unified definition of a substrate, which we struggled with a lot last year, to the (perhaps equally hard or harder?) goal of agreeing on key dimensions substrates may or may not have (perhaps in line with those, like Basman, who argue that several properties are optional rather than required). I would love to hear more of the authors' vision of their own continuum, and how we might situate work such as theirs, i.e., "not a fully-realized substrate", in this hypothetical space. 3. I was not really surprised to read that, amongst all people who heard about Allio or watched it in action, software developers were the most surprised ones. I totally agree with the authors' point about the role of technical education. I wonder whether they have suggestions for changes in, e.g., standard courses in programming or engineering curricula, that would be easy for professors to implement and effective in helping students broaden their view of software beyond desktop/mobile/web applications built on memory silos. One idea that keeps coming up in my head, my discussions with colleagues and some of what I read (such as Tomas Petricek's essay on the value of reconstructing old software) is that we should talk more about systems and technical solutions that were never used at scale or disappeared, which might teach us something about how much more diverse software ecosystems may be. Perhaps this would be easier to do if recreating/evaluating/adapting systems from the past had more "academic value" than it currently has? 4. Although I really like the "global search-and-replace tool" example of why Allio might be useful, I wish the authors could provide more examples of use cases that they envision. For example, I would really like to know more about the cross-application node-and-wire system depicted in Figure 1! 5. A different but related question: assuming some of these visions cannot be (fully) implemented because of current limitations, either in Allio itself or in the API it relies on (such as what is exposed/permitted by accessibility trees), what minimal changes would yield maximal potential (in the line of the concept of "floss moves" that Trividic describes in his submission to last year's workshop). 6. Another related question: In practice, how much of a problem was the lack of semantics in accessibility trees? When I was reading the part of the paper that mentions this problem, I could not help but think about early work on AI and/or programming-by-example techniques (such as inferring places, dates, etc., from plain text), as well as more recent work, such as my own work on Lorgnette, in which I propose to let programmers freely bind interactive tools to arbitrary pieces of text by describing the pattern and writing uni- or bi-directional mappings themselves (Gobert and Beaudouin-Lafon, 2023). I would also not be surprised if models such as LLMs and agentic AI systems, which, apparently, already rely on accessibility trees (is there any public, citable evidence of that?), could help solve the problem of giving semantic meaning to accessibility trees. |
| Review 2 | |
| PC member | Jonathan Edwards |
| Overall evaluation |
3: strong accept
This paper proposes that substrates can leverage accessibility APIs legally mandated in existing
platforms to interconnect and customize isolated applications. This is a great actionable idea
that should be shared with the substrates research community.There is a graveyard full of cross-application synthesis initiatives (applescript, opendoc, OLE, CORBA, etc). The paper points out the advantage of accessibility APIs is legal/regulatory requirements backed by political advocacy organizations. The paper also acknowledges this strength comes with limitations: “the demand was for machine-readable interfaces for assistive technology, not for general-purpose composition, and the technology was shaped accordingly.” My one criticism is that Section 5 veers into (unnecessary) politics worrying about “exploiting” the accessibility movement. It might be more productive to channel these worries into factual questions. Has the accessibility movement actually expressed such concerns? Wouldn’t they welcome allies? Have they objected to the related use of these APIs by AI agents? The assertion that “accessibility _is_ malleability” seems to contradict the statement about limitations quoted above. Perhaps 'ought' is meant rather than 'is'? Overall I enjoyed reading this paper, encountering many statements that made me stop and think and quote in my notes. This paper should stimulate a healthy conversation at the workshop. |
| Review 3 | |
| PC member | Gilad Bracha |
| Overall evaluation |
The paper discusses Allio, a tool that uses accessibility APIs to modify mainstream software in
impressive but limited ways. The technical point is quite fascinating, but the paper's main
concern is the socio-economic lessons to be drawn from this. The abstract does a great job of summarizing the key points. "everyone whose needs diverge from this imagined subject is disabled by the software environment" - I'll put it more strongly. Everyone, period, is disabled by the software environment - because the imagined subject doesn't exist. The people who build mainstream software don't want to use it in that form - they expect it is for some "dumb end-user". They likely prefer the Unix shell, git and similarly awful schemes (see the Unix haters handbook). "the commodity form actively selects against these properties because they dissolve the boundaries on which exchange value depends." Certainly. But that sentence makes it sound like some conspiracy. The paper itself indicates that the problem is structural. That distinction could be made clearer. In my view, there's more to it than that. The folks that dominate the construction these systems simply have no talent for HCI. They are themselves disabled (or at least impaired) in that sense. So poor usability and malleability are the path of least resistance. As the paper says, someone has to care and expend effort to make it otherwise. My point is, that the developers of the software don't really care (or even know) how to do better. That too may be structural, in a different way: the detail-oriented personalities that are good at coding, are often exceptionally bad at design and usability. Speaking of Parnas etc. : Modularity is necessary for complexity management and correctness, not just packaging. But it need not contradict malleability. After all, mainstream software does quite poorly on modularity as well. Reflective solutions can bypass standard module boundaries and access restrictions when needed. That said, I agree that doing that adds complexity, and it is very rarely done properly. "substrates programme’s central challenge as political-economic: " And yet "This does not mean that substrate researchers must become political organizers." Very relieved to hear. Politics corrupts all it touches. It is crucial we keep it out of science. But the tension between these statements is never truly resolved. You say: "It means that the question of who sustains substrate infrastructure and through what social mechanisms is as important as the question of what properties it should have. " What are we to make of this? That politicking works (see Stallman)? Overall, I am very uneasy with the political-sociological bent here. |
| Review 5 | |
| PC member | Luke Church |
| Overall evaluation |
This is a delightful work in multiple ways. - The accessibility stance draws attention of a wide audience to the nature of the power relationship they have with software, and the way that this emerges in a provocative and persuasive way - The way in which the work is contextualised into other work at Substrates really helps build a sense of it position, but also draws attention to what our current understanding of substrates does and does not effectively describe - There are some real gems in it: " The Parnas feedback loop — mode of production inscribing itself in tools, tools reproducing the mode of production — operates at the level of individual cognition as well as industry structure." Very true. - I also share authors concern about that the socio-political manoeuvre being contemplated is not without risk. I know basically nothing about the history of the accessibility infrastructure, and so my apologies if the authors have already done this, or are in fact part of the community that created these technologies - but I wonder what activists in the community feel about this work? Beginnings are such delicate times for building these alliances in a way that build mutually beneficial trajectories. Perhaps the note of caution I would have is that this could be read as saying to a global population "you are being disabled by your software". That is a message I would send with caution, partially because many may find it insulting as they read it through the lens of it being about their bodies, not about the system, and partially because of concerns you directionally discuss that it may inhibit the support for groups that do have different needs for whom their individual labels have had some utility. Oh and.. "This does not mean that substrate researchers must become political organizers." Are you sure? We choose to differentially enable and disable different positions. We often carry normative views "openess good, closedness bad". Perhaps some would argue this is pure aesthetics, but I'm not sure of what acts of design aren't political. Perhaps we don't need to be organizers. Perhaps it's more a matter of degree and researcher identity than category. It's a lovely provocative piece of writing and thinking. |